Evaluating Digital Math Tools in the Field

Many school districts have adopted digital tools to supplement or replace teacher-led instruction, usually based on the premise that these tools can provide more personalized or individualized experiences for students and at lower cost. Rigorously evaluating whether such initiatives promote better student outcomes in the field is difficult as most schools and teachers are unwilling to enforce rigorous study designs such as randomized control trials. We used study designs that were feasible in practice to assess whether two digital math tools, eSpark and IXL, were associated with improvements in 3rd – 6th grade student test scores in math. We also investigated the resource requirements and costs of implementing eSpark and IXL to assess whether these tools represent a valuable use of resources. We find that while IXL is substantially less costly to implement than eSpark, its use is not significantly associated with students’ math performance

Providing Comprehensive Educational Opportunity to Low-Income Students. Part 4: What Are the Social and Economic Returns?

Approximately half of all New York City public school students who live in families with incomes less than 185% of the federal poverty level (FPL) do not graduate from high school. These dropouts are much less likely to achieve economic self-sufficiency in adulthood. This creates both a fiscal and social burden. Rothstein, Wilder and Allgood (2011) propose a set of publicly funded comprehensive services to supplement academic services that could effectively raise the educational attainment levels of low income children to levels associated with middle class children. This report, the fourth in a five part feries, calculates the fiscal and social impact of raising the attainment levels of these students in order to assess whether the costs of such services are justified by the benefits. Specifically, the report estimates the economic costs and benefits attributable to a single cohort of 37,000 12th grade students from families with incomes below 185% FPL who are currently finishing up in New York City public schools. It calculates the net fiscal contributions by education level per individual. These contributions are tax revenues, minus government expenditures on healthcare, the criminal justice system, welfare programs, and school/college. The report finds that New York City and State combined spend $82,000 (in lifetime present value terms) on each high school dropout on healthcare, criminal justice, welfare, and education while receiving only$45,000 in tax revenues. This results in a deficit of $37,000 for each dropout. In contrast, for students earning a BA or higher degree, the city and state expenditures amount to$55,000 while tax revenues are $143,000, resulting in a balance of$88,000 in favor of the government. The impact for the federal government is also substantial with high school dropouts costing $1,360 when federal expenditures are netted against federal tax revenues, while students earning a BA or higher degree contribute$347,460 in lifetime present value terms. Evidently dropouts create a greater fiscal burden for the state and city than for the federal government, but the federal government gains dramatically more as education levels increase. The economic analysis provided in this report indicates that provision of all support programs in the comprehensive approach outlined by Rothstein, Wilder, and Allgood (2011) would require a significant outlay by state and local governments above and beyond what is currently expended. However, the resulting estimated 9% rate of return on investment compares favorably with existing forms of government debt, suggesting that a feasible strategy would be to finance the costs of a comprehensive support program for low-income children through the issuance of long-term municipal bonds. Philanthropic foundations could also help fund these social investments by providing grants to service providers that would be replaced by public funds if target outcomes were met

Cost-Effectiveness Analysis of Early Reading Programs: A Demonstration With Recommendations for Future Research

We review the value of cost-effectiveness analysis for evaluation and decision making with respect to educational programs and discuss its application to early reading interventions. We describe the conditions for a rigorous cost-effectiveness analysis and illustrate the challenges of applying the method in practice, providing examples of programs for which we have estimated costs, but find effectiveness data lacking in comparability. We provide a demonstration of how cost-effectiveness analysis can be applied to two early reading programs: the Wilson Reading System and Corrective Reading. We use existing effectiveness data from an experimental evaluation in which the programs were implemented under similar conditions and the use of common outcomes measures for both programs yielded data that are comparable. We combine these data with cost data we collected using the ingredients method to calculate cost-effectiveness ratios for the alphabetics domain. A complete picture of the relative cost-effectiveness of each program could be provided if effectiveness metrics were available for fluency, vocabulary, and comprehension. We highlight the obstacles to applying cost-effectiveness analysis more frequently and recommend strategies for improving the availability of the requisite data

Cost-effectiveness Analysis of Interventions that Improve High School Completion

This report demonstrates the methods of cost-effectiveness analysis as applied to several educational programs that have been shown to improve the rate of high school completion

Improving Early Literacy: Cost-Effectiveness Analysis of Effective Reading Programs

This study is a cost-effectiveness analysis of seven early literacy programs that have all been previously identified as effective at improving reading outcomes for students in Grades K-3. We use the ingredients method to collect cost data for each program and compare the cost-effectiveness of programs serving students in the same grade level

A core outcome set for pre‐eclampsia research: an international consensus development study

Objective To develop a core outcome set for pre‐eclampsia. Design Consensus development study. Setting International. Population Two hundred and eight‐one healthcare professionals, 41 researchers and 110 patients, representing 56 countries, participated. Methods Modified Delphi method and Modified Nominal Group Technique. Results A long‐list of 116 potential core outcomes was developed by combining the outcomes reported in 79 pre‐eclampsia trials with those derived from thematic analysis of 30 in‐depth interviews of women with lived experience of pre‐eclampsia. Forty‐seven consensus outcomes were identified from the Delphi process following which 14 maternal and eight offspring core outcomes were agreed at the consensus development meeting. Maternal core outcomes: death, eclampsia, stroke, cortical blindness, retinal detachment, pulmonary oedema, acute kidney injury, liver haematoma or rupture, abruption, postpartum haemorrhage, raised liver enzymes, low platelets, admission to intensive care required, and intubation and ventilation. Offspring core outcomes: stillbirth, gestational age at delivery, birthweight, small‐for‐gestational‐age, neonatal mortality, seizures, admission to neonatal unit required and respiratory support. Conclusions The core outcome set for pre‐eclampsia should underpin future randomised trials and systematic reviews. Such implementation should ensure that future research holds the necessary reach and relevance to inform clinical practice, enhance women's care and improve the outcomes of pregnant women and their babies

Resource Requirements and Costs of Developing and Delivering MOOCs

Given the ongoing alarm regarding uncontrollable costs of higher education, it would be reasonable to expect not only concern about the impact of MOOCs on educational outcomes, but also systematic efforts to document the resources expended on their development and delivery. However, there is little publicly available information on MOOC costs that is based on rigorous analysis. In this article, we first address what institutional resources are required for the development and delivery of MOOCs, based on interviews conducted with 83 administrators, faculty members, researchers, and other actors in the MOOCspace. Subsequently, we use the ingredients method to present cost analyses of MOOC production and delivery at four institutions. We find costs ranging from $38,980 to$325,330 per MOOC, and costs per completer of $74-$272, substantially lower than costs per completer of regular online courses, by merit of scalability. Based on this metric, MOOCs appear more cost-effective than online courses, but we recommend judging MOOCs by impact on learning and caution that they may only be cost-effective for the most self-motivated learners. By demonstrating the methods of cost analysis as applied to MOOCs, we hope that future assessments of the value of MOOCs will combine both cost information and effectiveness data to yield cost-effectiveness ratios that can be compared with the cost-effectiveness of alternative modes of education delivery. Such information will help decision-makers in higher education make rational decisions regarding the most productive use of limited educational resources, to the benefit of both learners and taxpayers